Improvements in big data analytics and cloud computing in recent years have given rise to systems and infrastructures that enable various entities to host or rent processing clusters to analyze and process data. In a large-scale computing cluster, various big data processing jobs may include a sequence of multiple stages or operations, in which intermediate data is generated after each stage, but the intermediate data may not be saved or stored for subsequent use during subsequent processing jobs.
The above information discussed in this Background section is only for enhancement of understanding of the background of the described technology and therefore it may contain information that does not constitute prior art that is already known to a person having ordinary skill in the art.